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ABSTRACT: PERSONAL IMAGING SYSTEM WITH 
VTEWFINDER AND ANNOTATION MEANS 

A novel means and apparatus for personal documentary photography and videog- 
raphy is described. In particular, a personal camera with viewfinder means and a 
personal video annotation system is introduced. The camera system integrates the 
process of making a personal handwritten diary or the like, with the capture of video, 
from an optimal point of vantage and camera angle. This gives rise to a new form 
of personal diary, as well as a new genre of documentary video. Video of a subject 
such as an official behind a counter may be captured by a customer or patron of an 
establishment, in such a manner that the official cannot readily determine whether 
or not video is being captured with the handwritten notes or annotations. 
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Patent Application 
of 

Steve Mann 
for 

PERSONAL IMAGING SYSTEM WITH VIEWFINDER AND 
ANNOTATION MEANS 

of which the following is a specification: 

FIELD OF THE INVENTION 

The present invention pertains generally to a new photographic or video means and 
apparatus typically comprising a hand-held portable electronic camera system with 
viewfinder means and electronic pen-based annotation means. 

BACKGROUND OF THE INVENTION 

In photography (and in movie and video production), it is often desirable to cap- 
ture events in a natural manner with minimal intervention or disturbance. In the 
present invention, the scenario to be considered is that of face-to-face conversation 
between two individuals, where one of the individuals wishes to make an annotated 
video diary of the conversation without disrupting the natural flow of the conver- 
sation. In this context, it is desirable to create a personal video diary or personal 
documentary, or to have some kind of personal photographic or videographic memory 
aid that forms the visual equivalent of what the electronic organizers and personal 
digital assistants do to help us remember textual or syntactic information. 

Current state-of-the-art photographic or video apparatus creates a visual dis- 
turbance to others and attracts considerable attention on account of the gesture of 
bringing the camera up to the eye. Even if the size of the camera could be reduced 
to the point of being negligible (e.g. no bigger than the eyecup of a typical cam- 
era viewfinder, for example), the very gesture of bringing a device up to the eye is 
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unnatural and attracts considerable attention, especially in establishments such as 
department stores where photography is often prohibited. However it is in these very 
establishments in which a customer may wish, for example, to have a video record of 
the clerk's statement of the refund policy or the terms of a sale. Just as department 
stores often keep a video recording of all transactions (and often even a video record- 
ing of all activity within the establishment, sometimes including a video recording of 
customers in the fitting rooms), the goal of the present invention is to assist, for ex- 
ample, a customer who may wish to keep a video record of a transaction, interaction 
with a clerk, manager, refund explanation, or the like. 

Although there exist a variety of covert cameras such a camera concealed beneath 
the jewel of a necktie clip, cameras concealed in baseball caps, and cameras concealed 
in eyeglasses, these cameras tend to produce inferior images, not just because of the 
technical limitations imposed by their small size, but, more importantly because they 
lack a viewfinder means (e.g. means of viewing the image to adjust camera angle for 
best composition). Because of the lack of viewfinder means, the subject matter is 
not necessarily centered well in the viewfinder, or even captured by the camera, and 
thus these covert cameras are not well suited to personal documentary or for use in 
a personal photographic/videographic memory assistant. 

A wearable camera system (called WearCam) has been proposed in, for example, 
IEEE Computer, Vol 30, No 2, or equivalently http://wearcam.org/ieeecomputer.html 
(see also http://wearcomp.org/wearpubs.html). Unfortunately embodiments of the 
WearCam invention typically require special preparation such as special clothing, and 
special wiring of the body a-priori. Such systems will work for those dedicated to be- 
ing constantly wired with various computational and sensory apparatus incorporated 
into special clothing, but there are a good many individuals who may wish to obtain 
the same convenience without a change of lifestyle. 

Accordingly, what is proposed is a camera and viewfinder means for unobtrusively 
capturing video of exceptionally high compositional quality and possibly even artistic 
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merit, and some embodiments of this invention that are hand-held rather than body- 
worn. 

In addition to the fact that covert versions of the apparatus can be used to create 
investigative documentary videos having very good composition, the device need not 
necessarily be covert. Instead, it may merely be designed so that the subject of the 
picture or video cannot readily determine whether or not the apparatus is in use 
recording. Just as department stores and the like often have dark domes on their 
ceilings, so that customers do not know whether or not there are cameras in the 
domes (or which ones have cameras and even which way the cameras are pointed if 
there are cameras in the domes), the proposed invention creates a situation in which a 
department store clerk or the like will not know whether or not a customer's personal 
memory assistant is recording video. This uncertainty is created by having the camera 
positioned so that it will typically be pointed at a person at all times, whether or not 
it is actually being used. 

BACKGROUND OF THE INVENTION WITH RESPECT TO PRIOR 
ART: The invention is related to the Personal Digital Assistant! (PDA) such as the 
Palm Pilot, the so-called "Paper-Based AudioNotebook" (Lisa J. Stifelman Aug- 
menting Real-World Objects: A Paper-Based Audio Notebook CHF96 Conference 
Companion, pp. 199-200, April 1996), and the general ideas of pen-based comput- 



There is also an important connection to portable computers, mobile computing, 
and wearable computers (See for example, IEEE Computer, Vol 30 No 2). 



or a combination clipboard and pen-based computer input device, hereafter referred 
to as "clipboard" (whether it be an electronic organizer, PDA, or a real pen and 



ing. 



DESCRIPTION OF THE INVENTION 



A typical embodiment of the invention comprises a hand-held pen-based computer 



paper clipboard into which some electronic devices have been added). 
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A camera is built into the clipboard with the optica] axis of the lens facing the 
direction from bottom to top of the clipboard, so that during normal face-to-face 
conversation, the person holding the clipboard will tend to point the camera at the 
other person while taking written notes of the conversation. In this manner, the 
intentionaJity (whether or not the person taking written notes is intending to point 
the camera at the other person) is masked by the fact that the camera will always 
be pointed at the other person by virtue of its placement in the clipboard. Thus the 
camera lens opening need not necessarily be covert, and in fact, could be deliberately 
accentuated (e.g. made more visible) if desired. To understand why it might be 
desirable to make it more visible, one can look to the cameras in department stores, 
which are often placed in large dark smoked plexiglass domes. In this way they are 
neither hidden nor visible, but rather, they serve as an uncertain deterrent to criminal 
conduct. While they could easily be hidden inside smoke detectors, ventillation slots, 
or small openings, the goal of the dome is to make the camera conceptually visible 
yet completely hidden. In a similar manner, a large lens opening on the clipboard 
may, at times, be desirable, so that the subject will be reminded that there could be a 
recording, but will be uncertain as to whether or not such a recording is actually taking 
place. Alternatively, a large dark shiny plexiglass strip, made from darkly smoked 
plexiglass (typically 1cm high and 22cm across) is installed across the top of the 
clipboard, as a very subtle yet visible deterrent to criminal behaviour and disrespect. 
One or more miniature cameras are then installed behind the dark plexiglass, looking 
forward through it. In other embodiments, a camera is installed in a PDA, and then 
the top of the PDA is covered with dark smoky plexiglass. 

The clipboard is fitted with an electronic display system which includes the capa- 
bility of displaying the image from the camera, so that it can serve as a viewfinder 
for aiming the camera at the subject. Moreover, the display is typically constructed 
so that it is visible only to the user of the clipboard, or, at the very least, so that the 
subject of the picture cannot readily see the display. Such concealment of the display 



4 



CA 02237939 1998-06-29 



may be accomplished through the use of a honeycomb filter placed over the display. 
Such honeycomb filters are common in photography, where they are placed over lights 
to make the light sources behave more directionally. They are also sometimes placed 
over traffic Lights where there is a wye intersection, so that the lights can only be seen 
from one direction, for example, so that the traffic lights do not confuse drivers on 
another branch of a wye intersection that faces almost the same way. Alternatively, 
the display may be designed to provide an inherently narrow field of view, or other 
barriers may be constructed to prevent the subject from seeing the screen. 

The function of the camera is integrated with the clipboard so that textual in- 
formation, as well as drawings, may be stored in a computer system, together with 
pictures or videoclips (herafter still pictures and segments of video will both be re- 
ferred to as videoclips, with the understanding that a still picture is just a video 
sequence which is one frame in length). 

Since videoclips are stored in the computer together with other information, these 
videoclips may be recalled by an associative memory working together with that 
other information. Thus tools like the UNIX "grep" command may be applied to 
videoclips by virtue of the associated textual information which typically resides as a 
videographic header. In this way, for example, one could grep for the word "meijer", 
and may find various videoclips taken during conversations with clerks in the Meijer 
department store. Thus such a videographic memory system may give rise to a 
memory recall of previous videoclips taken during previous visits to this particular 
department store, provided that one has been diligent enough to write down (e.g. 
enter textually) the name of the department store upon each visit. 

Videoclips are typically time-stamped (e.g. there exist file creation dates) and 
GPS-stamped (e.g. there exists global positioning system headers from last valid 
readout) so that one can search on setting (time-f place). 

Thus the video clipboard may be programmed so that the act of simply taking 
notes causes previous related videoclips to play back automatically in a separate 
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window (in addition to the viewfinder window which should always remain active for 
continued proper aiming of the camera). Such a video clipboard may, for example, 
assist in a refund explanation by providing the customer with an index into previous 
visual information to accompany previous notes taken during a purchase. This system 
is especially beneficial when encountering department store representatives who do 
not wear name tags and who refuse to identify themselves by name (as is often the 
case when they know they have done something wrong such as disrespect a customer). 

SUMMARY OF THE INVENTION: OBJECTS AND ADVANTAGES 

It is an object of this invention to provide a method of positioning a camera to take 
a picture or video of a subject without the subject's knowledge or at least without 
the subject's being certain as to whether or not said picture or video is being taken. 

It is a further object of this invention to provide a means of recording a picture or 
video where the spatial extent (field of view) of the image may be ascertained without 
having to hold any device up to the eye. 

It is a further object of this invention to provide such a method of taking a picture 
or video where the tonal characteristics of the picture may be ascertained without 
having to hold any device up to the eye. 

It is a further object of this invention to provide such a method of taking a picture 
or video where no apparent difference in body movement or gesture between when a 
picture is being taken and when no picture is being taken is detectable by others. 

It is a further object of this invention to provide the user with a means of deter- 
mining the composition of the picture from a display device that is located such that 
only the user can see the display device, and so that the user can ascertain the com- 
position of a picture or take a picture or video and transmit image(s) to one or more 
remote locations without the knowledge of others in the immediate environment, or 
without others in the environment being certain whether or not such picture or video 
is being captured. 
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It is a further object of this invention to provide a means and apparatus for a user 
to capture a plurality of images of the same scene or objects, in a natural process 
of simply sliding a small object around on a countertop, and then have these images 
combined together into a single image of increased spatial extent, spatial resolution, 
dynamic range, or tonal fidelity. 

It is a further object of this invention to provide a camera with viewfinder means 
for collaboration between the user of the apparatus and one or more other persons 
at remote locations through the manipulation of virtual objects such as cursors, or 
computer graphics renderings displayed upon the camera viewfinder. 

It is a further object of this invention to allow multiple users of the invention, 
whether at remote locations or side-by-side, or in the same room within each other's 
field of view, to interact with one another through the collaborative capabilities of 
the apparatus. 

It is a further object of this invention to allow multiple users of the invention, at 
remote locations, to collaborate in such a way that a virtual environment is shared in 
which camera-based camera-tracking of each user results in acquisition of video and 
subsequent generation of virtual information being made available to the other(s). 

It is a further object of this invention to allow multiple users of the invention, 
at the same location, to colla borate in such a way that multiple camer a viewpo ints 
may be shared among the users so that they can advise each other on matters such 
as composition, or so that one or more viewers at remote locations can advise one or 
more of the users on matters such as composition or camera angle. 

It is a further object of this invention to allow multiple users ofthejnyention, at 
different locations, to collaborate on a project or endeavour tha^j^yjaoLpertain to 
photography or videography directly, but a projec t or endeavour ja eyertheless-that is 
enhanced by the ability for each person to experience the^ viewpoint of an other. 

It is a further object of this invention to allow one or more remote participants at 
conventional desktop computers or the like to interact with one or more users of the 
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invention, at one or more other locations, to collaborate on a project or endeavour that 
may not pertain to photography or videography directly, but a project or endeavour 
nevertheless that is enhanced by the ability for one or more users of the invention to 
either provide or obtain advice from or to an other in dividual at a remote location. 

It is a further object of this invention to facilitate a very close-up camera angle, 
from slightly below the subject of the picture or video, most notably so that the 
camera can be brought closer to the subject than would be acceptable for the body of 
the camera operator, so that the personal space of the subject can be violated more 
effectively than with a body-worn came ra such as that of the WearCam invention. 

It is a further object of this invention to facilitate the creation of a database 
comprising video records linked to associated hand-written notes. 

It is a further object of this invention to facilitate the creation of video records 
linked to associated capture of handwriting and other information in and around the 
immediate environment. 

SUMMARY OF THE INVENTION: Informal review of what the new 
invention does 

One aspect of the invention allows the user to take notes with pen and paper (or 
pen and screen) and continuously recor<d_video together with the written notes. Even 
if there is insufficient memory to capture a continuous video recording, the invention 
can be designed so the user will always end up with the ability to produce a picture 
from something that was seen a couple jrfimnutes ago. This may be useful to everyone 
in the sense that we may not want to miss a great.photo.opportunity, and often great 
photo opportunities only become known to usjifter we have had time to think a bout 
something we previously saw. At the very least, if for example, a department store 
owner or manager becomes angry and insulting to the customer, the customer may 
retroactively record the event, by opening a circular buffer. 

Such an apparatus might also be of use in personal safety. Although there are a 
growing number of video surveillance cameras installed in the environment allegedly 
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for "public safety", there have been recent questions as to the true benefit of such 
centralized surveillance infrastructures. Most notably there have been several exam- 
ples in which such centralized infrastructure has been abused by the owners of it (as 
in roundups and detainment of peaceful demonstrators). Moreover, "public safety" 
systems may fail to protect individuals against crimes committed by members of the 
organizations that installed the systems. Accordingly, embodiments of the invention 
often implement the storage and retrieval of images by transmitting and recording 
images at one or more remote locations. In one embodiment of the invention, images 
were transmitted and recorded in different countries, so that they would be difficult 
to destroy, in the event that the perpetrator of a crime or other misconduct might 
wish to do so. 

The invention allows images to be captured in a natural manner, without giving 
an unusual appearance to others (such as an abusive official). 

Moreover, as an artistic tool of personal expression, the apparatus allows the user 
to record, from a new perspective, experiences that have been difficult to so record 
in the past. For example, a customer might be able to record an argument with a 
fraudulent business owner from a very close camera angle. This is possible because 
a clipboard may be extended outward toward the person without violating personal 
space in the same way as might be necessary to do the same with a camera hid- 
den in a tie clip, .baseball cap, or sunglasses. Since a clipboard may extend outward 
from the body, it may be placed closer to the subject than the normal eye viewpoint 
in normal face-to-face conversation. As a result, the camera can capture a very 
close-up view of the subject. Such a view is often more desirable than the view ob- 
tainable from a telephoto lens located in eyeglasses (e.g. telephoto embodiment of the 
WearCam invention) because the close-up view also provides a dramatic and power- 
ful composition, from underneath the subject's face, and is therefore most suitable, 
cinematographically, for capturing the menacing nature of a corrupt or disrespectful 
official. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The invention will now be described in more detail, by way of examples which 
in no way are meant to limit the scope of the invention, but, rather, these examples 
will serve to illustrate the invention with reference to the accompanying drawings, in 
which: 

FIG. 1 is a diagram of a simple embodiment of the invention in which there is 
a camera borne by a personal digital assistant (PDA) which has a separate display 
attached to it to function as a viewftnder for the camera. 

FIG. 2 is a diagram of an alternate embodiment of the camera system in which a 
graphics tablet is concealed under a pad of paper and an electronic pen is concealed 
inside an ordinary ink pen so that all of the writing on the paper is captured and 
recorded electronically together with video from the subject. 

FIG. 3 illustrates a wrist watch embodiment of the invention in which a camera 
concealed inside the watch sends video to a covert body-worn computer system which 
transmits the video to a remote site where it is processed by one or more remote 
experts and advice from the remote expert(s) is displayed on a computer screen 
on the face of the watch together with time, date, and video from the camera (for 
viewfinder purposes). 

FIG. 4 depicts a close-up view of the wristwatch display, which happens to be 
the display of a body-worn computer running XFree86 on top of the Linux operating 
system, which facilitates the display of an "xclock" together with other material such 
as video from the camera (so the display can function as a viewfinder), and various 
menu items suitable for a wristwatch videotelephone. 

While the invention shall now be described with reference to the preferred em- 
bodiments shown in the drawings, it should be understood that the intention is not 
to limit the invention only to the particular embodiments shown but rather to cover 
all alterations, modifications and equivalent arrangements possible within the scope 
of appended claims. 
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In all aspects of the present invention, references to "camera" mean any device or 
collection of devices capable of simultaneously deterq aining ~a quaritityof light arriving 
from a plurality of directions and or at a plurality of locations, or determining some 
other attribute of light arriving from a plurality of directions and or at a plurality 
of locations. Similarly references to "display" shall not be limited to just television 
monitors or traditional televisions used for the display of video from a camera near or 



distant, but shall also include computer data dis play, means,xompi*ter data monitors, 
other video display devices, still picture display devices, ASCII text display devices 

. ' ^==: ■ — 

and the like. 



References to "video" shall include any signal that includes one or more streams 
of visible pictures, together with one or more audi o channels from one or m ore mi- 
crophones. Thus the term "video" may, for example, refer to a signal recorded from 
two cameras and several microphones arranged„in~an-array, to be later processed for 
beamforming purposes or to be processed in such a way_as_to project a nullspace 



toward unwanted audible interference such as the sound of a vacuum cleaner in the 



appliances section of a Sears department store during the shooting of a documenatary 
video there. The term "video" shall also be taken to include intermediate representa- 
tions of signals derived from cameras, microphones, and associated auxiliary sensors. 
Examples of such intermediate representations shall include, but not be limited to, 
recording of some number M of audio channels from some number N of microphones, 
where AT may be less than N. Examples of "video" may also include image represen- 
tations from multiple electromagnetic sensors represented as a single signal. 

DETAILED DESCRIPTION OF THE INVENTION WITH 
REFERENCE TO DRAWINGS 

Fig 1 shows an embodiment of the invention built into a PDA IQO^Avideo camera 
110 captures a view of a person standing in front of the user of the PDA and displays 



the image on an auxiliary screen 120, which may be easily concealed by the user's 
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hand while the user is writing or pertending to write o n the PDA's screen 130 with 



pen 140. In commercial embodiments of the invention the "auxiliary screen 120 may 



not be necessary, as it may be implemented as a window displayin g the camera's 
view on a portion of the main screen 130. Annotations made on screen 130 with pen 



140 may also be captured and stored together with videoclips from camera 110, so 
that there is a unified database in which the notes and annotations are linked with 



the video. An optioned second camera 150 may be used if the user wishes to make 
a video recording of himself/herself while recording another^person .with camera 110. 
In this way. botrTsldes of the conversation may he simultaneous ly recorded by the 



two cameras, so that the resulting recordings could be edited later, so that there 
could, for example, be a cut back and forth between the two cameras to follow the 
natural flow of the conversation. Such a recording might, for example, be used for an 
investigative journalism stofy^n corrupt department store owners who illegally chain 



their fire exits shut. In the early prototypes, an additional wire 160 was run up the 



sleeve of the user into a separate body worn pack 170 powered by its own battery pack 
172. __The_body_wom -pack typically "contains al»rnputer system 174 which houses 
video capture hardware and is' connected to a communications system 176 comprising 
packet radio terminal node controller (high level data link controller with modem) and 
radio, which typically establishes an Internet connection by way of antenna 178. The 
body- worn pack 170 wouldlikely disappear in the final embodiment of the invention 
which would contain the functionality of the pack 170 inside the hand-held device 

100. 

Fig 2 depicts an alternate embodiment of the invention depicted in Fig 1 in which 
the invention is built into an ordinary clipboard 200. Video camera 210 displays on 
a minature screen 220 mounted to the clipboard. A folded back piece of paper 230 
conceals the screen 220. The rest of the sheets of paper 231 are placed slightly below 
the top sheet 230, so that the user can write on them in a natural fashion. From 
the perspective of someone facing the user (the subject), the clipboard will have the 
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appearance of a normal clipboard in which the top sheet 230 appears to be part of 
the stack 231. Pen 240 is a combined electronic pen and real pen, so that the user 
can simultaneously write on the paper with real ink, as well as make an electronic 
annotation by virtue of a graphics tablet below the stack of paper 231, provided that 
the stack is not excessively thick. In this way there is a computer database linking the 
real physical paper with its pen strokes and the video recorded of the subject. From 
a legal point of view, real physical pen strokes may have some forensic value that the 
electronic material may not (e.g. if the department store owner asks the customer to 
sign something, or even just to sign for a credit card transaction, the customer may 
place it over the pad 231 and use the special pen 240 to capture the signature in the 
customer's own computer and index it to the video record). Wire 260 going from the 
clipboard, up the sleeve of the user, would be eliminated through construction of a 
self-contained video clipboard. 

Fig 3 depicts a wristwatch embodiment 300 of the invention depicted in Fig 1. 
Camera 310 points forward such that, for example, while a customer is wearing the 
wristwatch embodiment of the invention and is standing at a counter, he or she can 
place his or her arm naturally upon the counter and aim the camera 310 at the of- 
ficial behind the counter without appearing unusual. The video of the official seen 
by camera 310 is displayed on display unit 320, so t hatjisj)lay unit 320 functions as 
a viewfinder to assist the customer in aiming the camera 310. An optional second 
camera 350 may be included if the customer wishes to record himself/herself so that 
both sides of the conversation would be recorded. Cabling 360 runs from the cam- 
era(s) 310 (and 350) inside the wrist strap, emerging at the back of the wrist strap 
where it will be concealed from view by the official. Cabling 360 runs up the sleeve of 
the wearer of the watch, to an internet-connected body-worn computer system, the 
output of which runs back down cabling 360 out to the display 320. In this way video 
from camera 310 may be transmitted and recorded at remote sites, while the wearer 
of the wristwatch may be advised by a remote legal expert on the best approach for 
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dealing with the corrupt or disrespectful official. In future embodiments, the cabling 
360 may be eliminated, so that the unit would be either entirely self-contained , or 
would communicate wirelessly with a body-worn computer/repeater/internet gate- 
way station or a computer/repeater/internet gateway station carried in a briefcase or 
the like. 

Interaction with the wristwatch version of the invention depicted in Fig 3 may be 
done through a pen-based or touch-based interface to the screen, in the form of a 
so-called "pie menu" as described in Callahan, Hopkins, Weiser and Shneiderman, 
1988, or by using a "unistroke shorthand" as invented by Goldberg and Richardson, 
1993. 

Fig 4 depicts a natural choice of pie menu for a wristwatch display. Display 400 
is typically a computer screen with 480 pixels down and 640 pixels across, measuring 
approximately 0.7 inches on the diagonal. Upon display 400 is the image of a clock 
face, superimposed on top of a video signal from the camera. Time is displayed as a 
video picture, similar to the way that time is displayed on a UNIX workstation using 
the "xclock" program. Thus the "hands" of the clock 410 are realized by a video image 
in which pixels are darker (or lighter) in the region of the hands, and in this way the 
embodiment of the camera invention tells time. In the figure depicted here, the time is 
4:03. Thus the device truly looks like an ordinary wristwatch (although one in which 
the hands are displayed electronically) because it is in fact a wristwatch, among other 
things. It is natural for such a wristwatch to have a circle 420 displayed on the screen, 
and to have numbers 430 displayed around the periphery of said circle. In this way 
it is easier to tell time, and also the numbers may be assigned a secondar y jneaning 
(e.g. select "0" to stop recording, "4" to kill all processes and halt the processor, "7" 
to wake up the system from sleep mode, etc.). 

Since humans are quite good at telling~time, often the numbers are missing from 
many commercial wristwatches, and some wristwatches do not even have markings 
for each hour. Instead, we often rely on our heightend sense of visual acuity to discern 
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the angle of the hands upon the clockface. Thus it is no surprise that the clock menu 
is usable without paying much attention to the face of the clock. The user just needs 
to stroke the face of the clock in the direction desired (e.g. stroke the clockface in the 
2:00 direction to enter the number a 2"). 

The entry of numbers on a touch sensitive clockface in the context of the current 
invention may be done as vectors (e.g. with no regard to location, only regard to 
direction). Thus a stroke from left to right is regarded as the number tt 3" regardless 
of where the stroke begins or ends. A downwards stroke (e.g. from top to bottom) is 
regarded as the number "6" regardless of where the stroke begins or ends, and so on. 

Thus telephone numbers can be easily entered into the device, and similarly an 
alphabet 440 can be constructed much like the alphabet of an automated DTMF 
answering system used for voicemail and the like in telephony. 

Such a menu may be implemented with touchscreen, or alternatively small push- 
buttons around the periphery of the clock face may be used. Since there axe 12 
pushbuttons on a telephone, and there are also 12 hours on a clock face, there can be 
a one to one correspondence between the numbers of the clockface and those of the 
telephone. The hours 10:00 and 11:00 are used for the symbols "** and "#* of the 
telephone touchpad. 

The data entered by way of the clock face menu is typically combined with the 
video recording made from the scene. Of course the amount of information taken down 
may be less than that in the embodiments of the invention described in Fig 1 and 
Fig 2, but the clock face menu is sufficient for entering a department store manager's 
name, which may be appended to the video file header, so that later a large database 
of recorded video may be navigated using these short text headers. Of course GPS 
and file creation are also important as in the other embodiments of the invention. 
Moreover, due to direct contact between the watch and the body, the pulse (heart 
rate), as well as skin conductivity (sweatiness index) of the wearer may be determined 
and this information may be appended to or recorded with the video signals. This 
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may facilitate, for example, a future search through all video in which the wearer's 
heart rate exceeds a certain threshold. It has been found that when a department 
store manager is dishonest with respect to refund policies, or a clerk refuses to tell 
a customer his/her name, that the customer's heart rate increases dramatically, and 
the customer often sweats profusely. Thus this extra information can later help locate 
moments of tension in a previously recorded argument at the refund counter. 

BENEFITS OF THE INVENTION 

The vast proliferation of video surveillance throughout various establishments such 
as department stores, public parks, streets, and even changerooms (Newsweek, July 
17, 1995) is propagated with a promise to a better, safer world. However, represen- 
tatives and architects of this surveillance superhighway may continue to be corrupt. 
For example, although cameras reduce and deter shoplifting, there continue to be sit- 
uations where shopkeepers illegally chain shut fire exits, or refuse to be accountable 
in other ways (such as making promises with respect to refund policy and then failing 
to keep these promises later). Extreme situations include the murder of a customer 
suspected of (and falsely accused of) shoplifting by a department store owner (see, 
for example, the Latasha Harlins case) where the department store owner shot the 
customer in the back as she turned to walk out of the store. 

Thus one benefit of the invention is to provide personal safety, and to ensure 
accountability to those who might otherwise escape accountability. These benefits are 
especially useful in a totalitarian regime (a regime which wishes to know everything 
about others but reveal nothing about itself). A typical department store (where video 
surveillance is used extensively yet photography and video by customers is prohibited) 
is a prime example of where the proposed invention may assist in providing some 
degree of balance with respect to mutual accountability. 

Furthermore, the invention is useful as a new communications medium, in the 
context of collaborative photography, collaborative videography, and telepresence. 
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One way in which the invention can be useful for telepresence is in the creation of 
video orbits (collections of pictures that exist in approximately the same orbit of the 
projective group of coordinate transformations). A video orbit can be constructed 
using the clipboard embodiment in which a small rubber bump is made on the bottom 
of the clipboard right under the camera's center of projection. In this way, when the 
clipboard is rested upon a surface such as a countertop, it can be panned around 
this fixed point, so that video recorded from the camera can be used to assemble a 
panorama or orbit of greater spatial extent than a single picture. Similarly with the 
wristwatch embodiment, a small rubber bump on the bottom of the wristband allows 
the wearer to place the wrist upon a countertop and rotate the entire arm and wrist 
about a fixed point. Either embodiment is very well suited to shooting a high quality 
panoramic picture or orbit of an official behind a high counter, as is typically found 
at a department store, bank, or other organization. 

Moreover, the invention may perform other useful tasks such as functioning as a 
personal safety device and crime deterrent by virtue of its ability to maintain a video 
diary transmitted and recorded at multiple remote locations. As a tool for photo- 
journalists and reporters, the invention has clear advantages over other competing 
technologies. 

OTHER EMBODIMENTS 

From the foregoing description, it will thus be evident that the present invention 
provides a design for a hand-held or wrist-worn camera with viewfinder means. As 
various changes can be made in the above embodiments and operating methods with- 
out departing from the spirit or scope of the following claims, it is intended that all 
matter contained in the above description or shown in the accompanying drawings 
should be interpreted as illustrative and not in a limiting sense. 

Variations or modifications to the design and construction of this invention, within 
the scope of the appended claims, may occur to those skilled in the art upon reviewing 
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the disclosure herein. Such variations or modifications, if within the spirit of this 
invention, are intended to be encompassed within the scope of any claims to patent 
protection issuing upon this invention. 
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CLAIMS 

The embodiments of the invention in which I claim an exclusive property or priv- 
ege are defined as follows: 

1. A hand-held video clip board system comprising a flat clipboard, processor, 
an electronic display connected to an output of said processor, and a camera 
connected to an input of said processor, where said video clip board system 
comprises said display attached to said clipboard such that writing paper may 
be also attached to said clipboard to conceal said display from a person standing 
in front of the user of said clipboard when said clipboard is held in a normal 
position for writing, and where said camera will face said person standing in 
front of user of said clipboard when said clipboard is held in a normal posi- 
tion for writing, and where said video clip boaxd system includes means for 
simultaneously taking hand written notes and capturing video signals from said 
camera. 

2. A hand-held video clip board system as described in Claim 1 further comprising 
a graphics tablet where said graphics tablet is located below a pad of paper 
upon said clipboard, and a special pen where said special pen includes means 
of marking, in ink, a sheet of paper on said pad of paper and where said special 
pen may be simultaneously tracked by said graphics tablet. 

3. A hand-held video clip board system as described in Claim 1 further comprising 
a graphics tablet and graphics pen where said graphics pen includes means 
for electronic drawing and writing upon said graphics tablet, and where said 
graphics pen also includes real ink for writing on real paper while providing said 
electronic drawing and writing means. 

4. A videographic memory system including a hand-held video clip board system 
as described in Claim 3 where said video signals and data pertaining to said 
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electronic drawing and writing means are both stored each time the clipboard 
is used, and farther including means of electronic query of said data for corre- 
sponding handwritten notes or corresponding video signal recordings. 

5. A wristwatch videoconferencing system comprising a housing worn on the wrist, 
where said housing contains a display in a flat face of said housing, where said 
display is connected to an output of a processor, and where said housing also 
contains a camera connected to an input of said processor, where said display 
is responsive to an output of said camera. 

6. Means and apparatus for a wristwatch videoconferencing system comprising 
a thin housing worn on the wrist, where said housing contains a display in 
a flat face of said housing, where said display is connected to an output of a 
wirelessly networked body-worn processor, and where said housing also contains 
a camera connected to an input of said body- worn processor, where said body- 
worn processor is responsive to either an output of said camera, or an output 
of a camera worn by another wearer of an identical apparatus. 

7. Means and apparatus for a wristwatch videoconferencing system comprising a 
thin housing worn on the wrist, where said housing contains a display in a 
flat face of said housing, where said display is connected to an output of a 
processor, and where said housing also contains a camera connected to an input 
of said processor, where said camera is mounted in said housing such that when 
the wearer of said housing is looking at said display in a manner similar to 
the manner that one normally looks at a wristwatch, that said camera will be 
pointed at a person standing in front of said weaxer of said housing. 

8. A wristwatch video conversation capture system comprising means and appa- 
ratus described in Claim 7 where said camera is a first camera, and where said 
wristwatch video conversation capture system includes a second camera where 
said second camera is mounted in said housing such that when the wearer of 
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said housing is looking at said display in a manner similar to the manner that 
one normally looks at a wristwatch, that said second camera will be pointed at 
the face of said wearer. 

9. A wristwatch videoconferencing system comprising means and apparatus de- 
scribed in Claim 7 where said wristwatch videoconferencing system includes 
means for wireless transmission of the video signals from said camera to at least 
one remote location. 

10. A wristwatch videoconferencing system comprising means and apparatus de- 
scribed in Claim 7 where said processor is a computer system wirelessly con- 
nected to a computer network, and where said wristwatch videoconferencing 
system includes means of transmitting video signals to a remote location while 
interacting with one or more remote entities by way of said display. 

11. A wristwatch video recording system comprising means and apparatus described 
in Claim 7 where said processor is a computer system including means for record- 
ing video signals from said camera, and where said wristwatch video recording 
system includes means for interacting with said display to control the recording 
process. 

12. A wristwatch videotelephone comprising means and apparatus described in 
Claim 7 where said processor is a computer system wirelessly connected to a 
computer network, and where said wristwatch videotelephone system includes 
means of transmitting video signals to a remote location while interacting with 
said computer system by way of said display. 

13. A wristwatch videotelephone as described in Claim 12 where said means of 
interacting with said computer system by way of said display comprises data 
entry using a pie menu. 
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14. A wristwatch videotelephone as described in Claim 12 where said means of 
interacting with said computer system by way of said display comprises data 
entry using a pie menu in which the wearer of the apparatus is presenLed with 
12 choices. 

15. A wristwatch videotelephone as described in Claim 14 where said 12 choices 
include the numerals 0 through 9 corresponding with the hours from midnight 
through 9:00. 

16. A wristwatch videotelephone as described in Claim 14 where said 12 choices 
include the numerals 0 through 9 corresponding with the hours from midnight 
through 9:00 and the symbols and corresponding with the hours of 
10:00 and 11:00. 

17. A video orbits capture system including a hand-held video clip board as de- 
scribed in Claim 1, where said video orbits capture system further includes 
means of limiting the movement of said clipboard to a constrained movement 
when placed upon a surface, where said constrained movement permits said 
camera to undergo approximately pure rotation, where the central axis of said 
rotation correspondes to a point close to the center of projection of said camera. 

18. A wristwatch videoconferencing system as described in Claim 5 where said 
wristwatch videoconferencing system further includes means of placement of 
said wrist upon a surface for rotation of said wrist approximately about a sin- 
gle point of rotation, and where said point of rotation is close to the center of 
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projection of said camera. 
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FIGURE 1 : PDA-BASED VIDEO RECORDING/ANNOTATION SYSTEM 



CA 02237939 1998-06-29 




FIGURE 2: VIDEO CLIPBOARD 
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FIGURE 3: WRISTWATCH VIDEOCONFERENCING AND COLLABORATIVE RECORDING SYSTEM 
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FIGURE 4: WRISTWATCH VIDEOCONFERENCING MENU 



